Python Testing With Pytest
What is Pytest
Pytest is a feature-rich, plugin-based ecosystem for testing your Python code. The pytest framework helps to write simple and scalable test cases for databases, APIs, or UI. It support complex functional testing for applications and libraries. It helps to write tests from simple unit tests to complex functional tests. This blog will help you to understand basics of pytest and some of the tools pytest provides to keep your testing efficient and effective.
Features
- Open source
- Simple and Easy syntax
- Skip tests
- Detailed info on failing assert statements.
- Can run a specific test or a subset of tests
- Automatically detect tests and functions
- Can run tests in parallel.
- Python 3.6+ and PyPy 3
- Modular fixtures
- Can run unittest (including trial)
Install Pytest
Run the following command in your command line
pip install -U pytest
Check the installed version of pytest
pytest --version
Basics of pytest
Lets learn the basic of Pytest with following examble. Create the test file pytest_test.py with the below code.
pytest_test.py
import pytest
def func(x):
return x + 1
def test_1():
assert func(3) == 4
def test_2():
assert func(4) == 4
Execute the test
python3 -m pytest pytest_test.py
Result
platform darwin -- Python 3.8.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: XXXX
collected 2 items
pytest_test.py .F [100%]
============================================================= FAILURES =============================================================
______________________________________________________________ test_2 ______________________________________________________________
def test_2():
> assert func(4) == 4
E assert 5 == 4
E + where 5 = func(4)
pytest_test.py:16: AssertionError
===================================================== short test summary info ======================================================
FAILED pytest_test.py::test_2 - assert 5 == 4
=================================================== 1 failed, 1 passed in 0.05s ====================================================
Here in pytest_test.py .
Dot(.) says success. F says failure
The [100%] refers to the overall progress of running all test cases. After it finishes, pytest then shows a failure report because func(4) does not return 4.
Run a subset of entire test
Specific test or a subset of tests can be run by Pytest. Create the test file pytest_test.py with the below code.
pytest_test.py
import pytest
@pytest.mark.set1
def test_method1():
x=5
y=6
assert x+1 == y,"test failed"
@pytest.mark.set2
def test_method2():
x=5
y=6
assert x+1 == y,"test failed"
Execute the test
python3 -m pytest -m set1 pytest_test.py
Result
======================================================= test session starts ========================================================
platform darwin -- Python 3.8.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: XXXX
collected 2 items / 1 deselected / 1 selected
pytest_test.py . [100%]
========================================================= warnings summary =========================================================
pytest_test.py:34
XXXX/pytest_test.py:34: PytestUnknownMarkWarning: Unknown pytest.mark.set1 - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
@pytest.mark.set1
pytest_test.py:41
XXXX/pytest_test.py:41: PytestUnknownMarkWarning: Unknown pytest.mark.set2 - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
@pytest.mark.set2
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================================== 1 passed, 1 deselected, 2 warnings in 0.01s ============================================
pytest -m set1 will run only test_method1()
Group multiple tests in a class
Grouping tests in classes can be beneficial for the following reasons :
- Test organization
-
Sharing fixtures for tests only in that particular class.
-
Applying marks at the class level and having them implicitly apply to all tests.
-
Each test has a unique instance of the class.
Create the test file pytest_test.py with the below code.
pytest_test.py
import pytest
class TestClass:
def test_one(self):
x = "hello"
assert "p" in x
def test_two(self):
x = "ok"
assert hasattr(x, "check")
Execute the test
python3 -m pytest pytest_test.py
Result
======================================================= test session starts ========================================================
platform darwin -- Python 3.8.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: XXXX
collected 2 items
pytest_test.py FF [100%]
============================================================= FAILURES =============================================================
_________________________________________________________ TestClass.test_1 _________________________________________________________
self = <pytest_test.TestClass object at 0x103e749a0>
def test_1(self):
x = "hello"
> assert "p" in x
E AssertionError: assert 'p' in 'hello'
pytest_test.py:24: AssertionError
_________________________________________________________ TestClass.test_2 _________________________________________________________
self = <pytest_test.TestClass object at 0x103df96a0>
def test_2(self):
x = "ok"
> assert hasattr(x, "check")
E AssertionError: assert False
E + where False = hasattr('ok', 'check')
pytest_test.py:28: AssertionError
===================================================== short test summary info ======================================================
FAILED pytest_test.py::TestClass::test_1 - AssertionError: assert 'p' in 'hello'
FAILED pytest_test.py::TestClass::test_2 - AssertionError: assert False
======================================================== 2 failed in 0.05s =========================================================
The first test passed and the second failed. Intermediate values in the assertion can helps to understand the reason for the failure.
Pytest fixtures
Fixtures are used when we want to run some code before every test method. So instead of repeating the same code in every test we define fixtures. Usually, fixtures are used to initialize database connections, pass the base , etc
A method is marked as a Pytest fixture by marking with @pytest.fixture
pytest_test.py
import pytest
@pytest.fixture
def abc():
a=2
b=4
c=8
return[a,b,c]
def test_a(abc):
z=5
assert abc[0]==z , "failed"
def test_b(abc):
z=4
assert abc[1]==z, "passed"
def test_c(abc):
assert abc[2]== abc[1], "failed"
Execute the test
python3 -m pytest pytest_test.py
Result
======================================================= test session starts ========================================================
platform darwin -- Python 3.8.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: XXXX
collected 3 items
pytest_test.py F.F [100%]
============================================================= FAILURES =============================================================
______________________________________________________________ test_a ______________________________________________________________
abc = [2, 4, 8]
def test_a(abc):
z=5
> assert abc[0]==z , "failed"
E AssertionError: failed
E assert 2 == 5
pytest_test.py:61: AssertionError
______________________________________________________________ test_c ______________________________________________________________
abc = [2, 4, 8]
def test_c(abc):
> assert abc[2]== abc[1], "failed"
E AssertionError: failed
E assert 8 == 4
pytest_test.py:68: AssertionError
===================================================== short test summary info ======================================================
FAILED pytest_test.py::test_a - AssertionError: failed
FAILED pytest_test.py::test_c - AssertionError: failed
=================================================== 2 failed, 1 passed in 0.05s ====================================================
Here
We have a fixture named abc(). This method will return a list of 3 values. We have 3 test methods comparing against each of the values.
Pytest xfail / skip tests
pytest_test.py
import pytest
@pytest.mark.skip
def test_add_1():
assert 100+200 == 400,"failed"
@pytest.mark.xfail
def test_add_2():
assert 15+13 == 100,"failed"
def test_add_3():
assert 3+2 == 6,"failed"
def test_add_4():
assert 3+2 == 5,"failed"
Execute the test
python3 -m pytest -v pytest_test.py
Result
======================================================= test session starts ========================================================
platform darwin -- Python 3.8.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /Library/Developer/CommandLineTools/usr/bin/python3
cachedir: .pytest_cache
rootdir: XXXX
collected 4 items
pytest_test.py::test_add_1 SKIPPED (unconditional skip) [ 25%]
pytest_test.py::test_add_2 XFAIL [ 50%]
pytest_test.py::test_add_3 FAILED [ 75%]
pytest_test.py::test_add_4 PASSED [100%]
============================================================= FAILURES =============================================================
____________________________________________________________ test_add_3 ____________________________________________________________
def test_add_3():
> assert 3+2 == 6,"failed"
E AssertionError: failed
E assert 5 == 6
E +5
E -6
pytest_test.py:92: AssertionError
===================================================== short test summary info ======================================================
FAILED pytest_test.py::test_add_3 - AssertionError: failed
======================================== 1 failed, 1 passed, 1 skipped, 1 xfailed in 0.06s =========================================
Here
- test_add_1 is skipped and will not be executed.
- test_add_2 is xfailed. These test will be executed and will be part of xfailed(on test failure) or xpassed(on test pass) tests. There won’t be any traceback for failures.
- test_add_3 and test_add_4 will be executed and test_add_3 will report failure with traceback while the test_add_4 passes.
Pytest parameterized test
The purpose of parameterizing a test is to run a test against multiple sets of arguments. We can do this by @pytest.mark.parametrize .
Create the test file pytest_test.py with the below code. Here the 3 arguments are passed to a test method. This test method will add the first 2 arguments and compare it with the 3rd argument.
pytest_test.py
import pytest
@pytest.mark.parametrize("input1,input2,output" ,[(5,5,10),(3,4,2)])
def test_add(input1, input2, output):
assert input1+input2 == output,"failed"
Here the test method accepts 3 arguments- input1, input2, output. It adds input1 and input2 and compares against the output.
Execute the test
python3 -m pytest -v pytest_test.py
Result
======================================================= test session starts ========================================================
platform darwin -- Python 3.8.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /Library/Developer/CommandLineTools/usr/bin/python3
cachedir: .pytest_cache
rootdir: XXXX
collected 2 items
pytest_test.py::test_add[5-5-10] PASSED [ 50%]
pytest_test.py::test_add[3-4-2] FAILED [100%]
============================================================= FAILURES =============================================================
_________________________________________________________ test_add[3-4-2] __________________________________________________________
input1 = 3, input2 = 4, output = 2
@pytest.mark.parametrize("input1,input2,output" ,[(5,5,10),(3,4,2)])
def test_add(input1, input2, output):
> assert input1+input2 == output,"failed"
E AssertionError: failed
E assert 7 == 2
E +7
E -2
pytest_test.py:76: AssertionError
===================================================== short test summary info ======================================================
FAILED pytest_test.py::test_add[3-4-2] - AssertionError: failed
=================================================== 1 failed, 1 passed in 0.05s ====================================================
Conclusion
Pytest will make your testing experience more productive and enjoyable. Pytest syntax is very simple and easy. With pytest simple and complex tasks require less code and can be executed through a variety of time-saving commands and plugins. It can run a specific test or a subset of tests and also skip the particular test. PyTest offers a core set of productivity features to filter and optimize your tests along with a flexible plugin system. Whether you have a huge unittest or you’re starting a new project from scratch, pytest has something to offer you.
Install PyTest and give it a try !
